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ABSTRACT 

The  process  of  registering  oblique  EO  imagery  simultaneously  acquired  from  the 
same  platform  as  a  SAR  image  is  reported  along  with  the  usefulness  of  such 
information.  The  geometry  of  the  images  is  different,  and  the  process  of  warping  the 
oblique  EO  image  to  the  nadir  or  plan  map  type  view  of  the  SAR  leads  to  information 
about  elevations  within  the  scene  which  cannot  be  obtained  from  either  of  the 
individual  sensors  without  the  inclusion  of  either  prior  knowledge  of  the  scene  or 
recognition  capabilities. 
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Registered  Oblique  EO  Imagery  and  Synthetic 
Aperture  Radar  (SAR) 


EXECUTIVE  SUMMARY 

Long  range  oblique  Electro-Optic  (EO)  images,  such  as  those  taken  from  the  side  of  an 
aircraft,  provide  pictures  which  we  can  generally  recognise.  Unfortunately  the 
geometry  of  this  imaging  process  makes  it  difficult  to  determine  the  relative  position  of 
objects  in  terms  of  range  since  the  scale  of  the  image  varies  as  objects  become  more 
remote.  Unless  the  objects  in  a  scene  can  be  identified  and  the  relative  scales  computed 
from  knowledge  of  their  actual  size,  then  it  is  not  possible  to  determine  the  relative 
positions  of  objects  within  a  scene. 

Airborne  Synthetic  Aperture  Radar  (SAR)  provides  an  image  in  all  weather 
conditions.  Objects  within  the  image  are  easily  located  since  the  image  has  the  same 
geometry  as  a  topographic  map;  however  the  resolution  of  the  SAR  image  is  less  than 
that  obtainable  with  an  EO  sensor  and  recognition  is  generally  more  difficult.  It  is 
possible  to  register  the  EO  image  with  a  plan  map  instead  of  a  SAR;  however  it  is  not 
envisaged  that  the  approaches  presented  in  this  paper  will  be  used  on  regions  where 
this  type  of  information  is  available,  so  the  fusion  of  plan  maps  and  oblique  imagery 
will  not  be  discussed. 


By  warping  the  EO  image  to  align  with  the  SAR  image  the  advantages  of  the  EO 
image,  with  regard  to  feature  recognition,  are  retained,  while  the  relative  location  of 
features  is  simplified  since  the  image  now  has  a  geometry  similar  to  a  topographic 
map.  In  the  process  of  warping  the  EO  image,  information  about  the  elevation  and 
structure  of  features  within  the  image  are  obtained.  This  elevation  information  cannot 
be  directly  obtained  with  either  of  the  sensors  individually.  Unfortunately  the 
matching  process  between  images  is  difficult  to  perform  automatically.  In  some  special 
cases  it  may  be  possible  to  determine  the  size  of  an  object  from  an  EO  image  using 
either  the  shadow  or  knowledge  of  the  size  of  other  objects  in  the  scene;  however  it  is 
impossible  to  determine  the  elevation  at  which  the  object  exists  without  the  additional 
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information  available  from  a  second  sensor  which  has  a  different  perspective  view  of 
the  scene. 


This  report  outlines  the  procedures  involved  in  warping  an  oblique  EO  image  to  a 
SAR  image  and  discusses  the  merits  of  this  procedure.  Automation  of  the  elevation 
extraction  procedure  is  also  discussed  and  a  simple  simulation  is  performed  to 
demonstrate  its  usefulness.  Unfortunately  the  real  SAR  information  that  was  available 
was  of  an  extremely  poor  quality  due  to  a  failure  in  the  equipment  and  no  additional 
trials  were  able  to  be  performed.  The  existing  real  images  did  indicate  that  some 
additional  assumptions  would  be  required  to  enable  processing  to  occur  automatically- 
the  nature  of  these  assumptions  will  vary  depending  on  the  nature  of  the  scene  and  it 
was  believed  that  advice  should  be  sought  regarding  the  application  of  the  approach 
before  further  work  should  occur  using  what  might  turn  out  to  be  invalid  assumptions 
about  the  image  type.  The  simulated  scenes  represent  scenes  which  would  require  very 
few  or  simple  assumptions  and  merely  demonstrate  that  automatic  processing  is  in  fact 
possible. 
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1.  Introduction 

Long  range  oblique  Electro  Optic  (EO)  imagery  represents  a  form  of  information 
gathering  which  is  remote,  controllable  (assuming  the  weather  cooperates)  and  reveals 
aspects  of  structure  that  are  different  from  that  of  other  remote  information  gathering 
processes,  such  as  satellite  or  high  altitude  aircraft.  The  imagery  is  however  difficult  to 
analyse  because  its  scale  varies  over  the  scene,  and  the  objects  within  the  scene  will 
appear  to  be  distorted  and  misplaced  between  two  images  taken  from  slightly  different 
locations.  The  EO  imagery  considered  in  this  report  considers  both  visible  and  Infrared 
(IR)  images. 

SAR  imagery,  especially  when  taken  from  the  large  oblique  angles  considered  in  this 
report,  produces  an  image  which  looks  very  much  like  a  nadir  view  image. 
Consequently  the  features  are  easily  associated  with  features  on  a  map.  The  images 
have  the  advantage  that  resolution  is  constant  with  range  from  the  sensor  and  that  they 
are  immune  to  the  effects  of  cloud  and  smog;  however  the  resolution  is  generally  lower 
than  that  obtainable  from  an  EO  sensor  and  significant  noise  and  clutter  can  occur. 

This  report  contains  an  evaluation  of  what  can  be  gained  from  the  combination  of 
oblique  EO  imagery  and  Synthetic  Aperture  Radar  (SAR).  This  provides  an  approach 
where  oblique  EO  and  nadir  images  can  be  obtained  simultaneously  from  the  same 
platform.  The  implication  of  Digital  Terrain  Models  (DTM)  on  this  work  is  also 
discussed. 

Section  2  discusses  the  procedures  used  in  warping  an  oblique  EO  image  to  a  nadir 
view,  and  explains  how  measurement  of  the  camera  and  line  of  sight  parameters  allow 
this  to  be  done  automatically.  Section  3  explains  how  the  different  perspectives 
obtained  in  the  SAR  and  EO  images,  taken  from  the  same  platform,  allow  elevation 
information  to  be  obtained.  Section  4  presents  the  reverse  situation  to  Section  3  where 
the  elevation  is  known  and  can  be  used  to  correct  the  distortion  in  the  image.  Section  5 
considers  the  combination  of  the  SAR  and  the  oblique  EO  image.  The  extraction  of  a 
DTM,  the  structure  of  buildings  and  the  improved  classification  possible  from  the 
combined  information  are  all  discussed.  Section  6  discusses  the  advantage  of 
representing  information  in  a  3D  form  rather  than  as  a  2D  representation  which  is 
subject  to  perspective  differences  in  different  images  of  the  same  scene.  Section  7  briefly 
discusses  some  resolution  issues  which  arise  from  the  use  of  two  sensors  with  different 
resolutions.  Finally  some  recommendations  for  future  work  are  given. 
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2.  Remapping  Long  Range  Oblique  Images 

Analysis  of  long  range  oblique  aerial  images  (see  Figures  9, 12  and  13  in  Appendix  B)  is 
difficult  since  the  scale  of  the  information  varies  across  the  image,  making  it  difficult  to 
compare  structures  in  different  parts  of  the  image  or  to  determine  the  geographical 
association  of  objects;  however  a  transformation  can  be  applied  to  the  image  to  equalise 
the  scale  of  the  detail  and  register  the  image  on  a  range-azimuth  projection  (see  Figure 
10,  14  and  15  in  Appendix  B).  In  the  oblique  imaging  situation  the  range-azimuth 
projection  provides  substantially  the  same  information  as  would  be  available  from  a 
nadir  view  of  the  scene  (see  Section  3).  The  scale  equalisation  applied  to  the  oblique  EO 
image  to  form  the  range-azimuth  projection  makes  the  determination  and 
measurement  of  information  in  the  scene  simple. 

The  transformation  can  be  determined  precisely  if  knowledge  of  the  imaging 
parameters  and  some  information  about  the  scene  is  known.  The  resultant  image  has 
the  disadvantage  that  elevation  information  is  smeared  in  the  range  direction.  If  the 
elevation  is  known  then  this  smearing  can  be  eliminated;  however  this  will  leave 
portions  of  the  image  blank  since  they  were  obscured  by  the  objects,  which  were 
smeared  in  range,  from  the  EO  sensor.  Alternatively  if  the  true  location  of  objects  is 
known,  then  the  elevation  of  the  objects  can  be  determined. 

The  transformation  of  the  image  to  a  range-azimuth  projection  occurs  by  remapping 
the  pixels  in  the  original  image.  The  transformation  is  nonlinear  but  is  relatively  simple 
to  obtain  since  it  can  be  derived  from  the  camera  and  line  of  sight  parameters  at  the 
time  the  image  was  acquired.  The  optical  image  is  represented  by  a  grid  of  square 
pixels  in  the  horizontal  (indexed  by  x)  and  vertical  (indexed  by  y)  directions.  Each  pixel 
represents  the  scene  in  an  arc  of  azimuth  and  elevation  from  the  camera;  however, 
although  the  pixel  grid  is  uniform  the  elevation  and  azimuth  viewed  by  each  pixel  is 
not. 
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Figure  1:  Relationship  between  pixel  position  and  the  azimuth  viewed  by  each  pixel. 


The  azimuth  and  elevation  (Figure  1)  relative  to  the  optical  axis  of  the  camera  can  be 
calculated  from 


az  =  tan  1 1  — 
f 


and  el  =  tan 


-if  y 


where  x  and  y  are  the  horizontal  and  vertical  distances  in  mm  from  the  centre  of  the 
image  in  the  imaging  plane  of  the  camera  and  / is  the  focal  length  of  the  camera  in  mm. 
To  allow  for  rotation  in  the  image  plane  (Figure  2)  about  the  optical  axis,  the  x  and  y 
positions  are  found  from  the  pixel  locations  xp  and  yp,  where  the  pixel  origin  is  in  the 
middle  of  the  image. 


3 


UNCLASSIFIED 


DSTO-TR-0117 


UNCLASSIFIED 


Figure  2:  Correction  for  rotation  about  the  optical  axis. 


cos0 
sin  9 


-sinO 

cos0 


xp  *  psize 
yp  *  psize 


psize  is  the  size  in  mm  of  each  pixel  in  the  focal  plane  of  the  sensor,  and  0  is  the  angle 
of  rotation  of  the  camera  about  the  optical  axis.  *  represents  multiplication.  To  align  the 
image  with  a  global  coordinate  system,  it  is  necessary  to  know  from  what  point  the 
image  was  obtained  and  the  direction  of  the  camera's  optical  axis. 
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Figure  3:  Projection  of  an  Oblique  EO  scene  on  a  Nadir  view. 


Images  which  have  been  mapped  to  a  range-azimuth  projection  consist  of  pixels 
which  are  indexed  by  range  and  cross  range.  The  range  represents  the  distance  of  a 
pixel  from  the  flight  path  of  the  aircraft  measured  along  a  line  perpendicular  to  the 
flight  path  of  the  aircraft.  The  cross  range  represents  the  distance  between  the  pixel  and 
an  origin  point  which  is  measured  along  a  line  parallel  with  the  flight  path  of  the 
aircraft  (see  Figure  3  and  4). 

The  range  and  cross  range  are  calculated  in  the  following  manner 

ra  =  rsin(azr  +  az) 
era  =  dist  -  r  cos(a  +  az. ) 


Figure  4:  Determination  of  range,  r,from  height  and  angle  information. 
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_  h-  groundheight(ra,cra) 
tan  (dec -el) 

ra  is  the  range  from  the  flight  path,  era  is  the  cross  range  or  distance  from  the  origin 
parallel  to  the  flight  path  (this  consists  of  2  parts,  the  distance  the  plane  has  travelled 
beyond  the  origin,  dist,  and  a  distance  caused  by  camera  pointing).  azc  is  the  angle 
between  the  flight  path  and  the  direction  in  which  the  camera's  optical  axis  was 
pointed,  dec  is  the  declination  of  the  camera  from  the  vertical,  h  is  the  height  of  the 
aircraft  and  groundheight(ra,  era)  is  the  height  of  the  ground,  and  any  objects  on  it,  at  a 
particular  range  and  cross  range.  In  the  distorted  images  above  the  ground  height  was 
not  known  and  was  assumed  to  be  constant. 

2.1.  Determining  Camera  Measurements 

It  is  often  difficult  to  directly  measure  the  parameters  used  in  the  above  procedure  and 
an  analysis  of  the  potential  errors  involved  is  performed  in  Appendix  A.  It  is  also 
possible  to  determine  the  parameters  accurately  after  image  acquisition  but  this 
requires  higher  level  knowledge,  and  control  points.  Control  points  are  features 
present  within  the  image  which  are  identifiable  in  both  the  EO  image  and  the  SAR,  or 
another  nadir  view  (another  sensor.  Digital  Terrain  Map  (DTM)1,  map  etc.).  If  the 
altitude  of  these  identifiable  control  points  is  known,  then  the  camera  parameters  can 
be  determined  by  estimating  and  then  iteratively  modifying  the  parameters  so  that  the 
magnitude  of  the  error  between  the  predicted  location  of  the  control  points  in  the  EO 
image  calculated  using  the  camera  parameters,  and  their  true  location  is  minimised. 
The  procedure  of  selecting  control  points  can  be  performed  manually  in  most  scenes 
but  will  be  difficult  to  do  automatically  in  all  situations.  Unfortunately  this  technique  is 
the  only  approach  which  will  lead  to  sufficiently  accurate  camera  parameters  to 
register  the  images,  or  to  determine  additional  information,  such  as  height,  to  a 
precision  approaching  the  resolution  of  typical  EO  systems. 


3.  Elevation  Extraction  Using  SAR  Information 

The  advantage  of  SAR  as  a  surveillance  system  is  that  it  offers  an  all  weather  standoff 
imaging  system.  Long  range  oblique  EO  imagery,  although  not  an  all  weather  system, 
can  provide  imagery  at  a  high  resolution  and  from  different  spectral  bands,  making 
classification  of  objects  within  the  image  simpler.  The  combination  of  information  from 
the  two  sensors  may  offer  some  improvements  over  the  independent  systems.  SAR 
images,  just  like  EO  images,  produce  shadows  which  enable  the  height  of  objects  to  be 
determined.  This  process  is  difficult  since  the  recognition  of  shadows  will  be  difficult 

1  A  DTM  is  a  digital  map  in  which  the  elevation  of  the  surface  is  represented  at  a  grid  of  points. 
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and  meaningless  unless  information  is  known  about  the  ground  over  which  the 
shadow  is  projected.  The  accuracy  of  this  information  will  be  equivalent  to  that  of  the 
SAR.  Interferometric  SARs[l]  probably  offer  a  better  means  of  automatically 
determining  height  information  than  the  fusion  of  EO  and  SAR;  however  the  result  will 
probably  be  less  accurate  than  that  obtainable  with  an  EO  sensor  simply  because  of  the 
high  noise  level  in  SAR  images. 

Combining  the  information  from  the  two  sensors  is  difficult  because  the  images  are 
different  projections  of  the  scene  (see  Figures  9  and  10).  The  two  sensors,  mounted  on 
the  same  platform  look  at  the  same  3D  scene  and  record  the  information  as  a  2D 
projection  of  that  scene  (Figure  5).  The  EO  imagery  records  an  azimuth  and  elevation 
projection,  while  the  SAR  records  an  azimuth  and  range  projection.  The  SAR  projection 
is  at  right  angles  to  the  projection  of  the  EO  sensor  (Figure  5).  An  image  with  the  same 
geometry  as  the  SAR  projection  could  have  been  obtained  by  placing  an  azimuth- 
elevation  sensor  at  right  angles  to  the  original  optical  axis.  The  SAR  image  is  slightly 
different  to  the  image  that  would  be  produced  by  placing  an  EO  sensor  at  the 
equivalent  azimuth-elevation  SAR  projection  point  (see  Figure  5),  since  the  SAR  detects 
all  the  surfaces  that  are  illuminated  and  so  detects  surfaces  which  are  hidden  from  the 
equivalent  azimuth-elevation  sensor  but  displays  them  as  if  they  were  visible  from  that 
point,  ie.  parts  of  the  image  which  are  not  illuminated  by  the  SAR  appear  transparent 
in  the  equivalent  azimuth-elevation  projection.  The  consequence  of  this  is  that 
although  a  stereo  pair  of  images  has  been  obtained,  some  of  the  problems  caused  by 
having  to  associate  one  side  of  an  object  in  one  image  with  another  side  of  the  object  in 
the  other  image  are  avoided  since  both  images  show  the  same  facets  of  objects,  but 
from  different  projections. 

The  angle  between  views  of  the  scene  is  large  compared  to  standard  stereo  images. 

This  has  the  advantage  that  high  accuracy  and  resolution  of  depth  is  obtained  but 
increases  the  difficulty  in  matching  the  two  images  together,  since  the  disparity  caused 
by  elevation  is  large.  The  matching  is  also  made  more  complicated  because  features 
and  noise  statistics  may  appear  to  be  different  in  different  parts  of  the  electromagnetic 
spectrum. 
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Equivalent  Azimuth/Elevation 
SAR  Projection  Point 


Figure  5:  Effective  views  of  a  scene. 


Mis-registration  between  a  registered  oblique  EO  image  under  the  assumption  that 
the  groundheight  is  constant,  and  a  nadir  view  image  (such  as  SAR),  can  be  directly 
attributed  to  the  ground  height2.  From  Figure  5  it  can  be  seen  that  the  building  top 
appears  in  the  oblique  image  at  a  greater  elevation  than  the  base;  if  it  was  assumed  that 
the  building  was  flat  (as  would  be  done  when  no  information  about  the  building  was 
available)  then  the  top  of  the  building  would  be  mapped  to  a  point  on  the  ground 
behind  it.  The  change  in  the  value  of  r,  A r ,  for  the  EO  sensor  compared  to  the  true 
position  (see  Figure  4)  can  be  expressed  as 

^  _  Agroundheight(ra,cra) 
tan  (dec  -el) 

where  Agroun dheighti  ra,  era)  is  the  difference  between  the  assumed  elevation  of  the 
building  when  performing  the  registration  of  the  oblique  EO  to  the  SAR  image  and  the 
actual  elevation.  The  value  that  is  obtained  is  not  the  height  of  the  building  but  the 

2  Ground  height  is  used  to  refer  to  the  height  of  the  ground  above  an  arbitrary  zero  elevation 
plane  (refer  to  Figure  4)  or  in  the  case  where  an  object  exists,  such  as  a  building,  then  ground 
height  refers  to  the  distance  between  the  top  of  the  building  and  the  zero  elevation  plane. 
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distance  between  the  zero  elevation  plane  and  the  point  of  the  building  in  the  EO 
image  used  to  perform  the  registration.  To  determine  the  height  of  the  building  the 
A groundheight  for  both  the  top  and  bottom  of  the  building  need  to  be  determined.  This 
range  error  appears  in  the  image  as  a  smearing  of  elevation  into  range3.  Conversely  if 
the  smearing  can  be  determined  then  the  true  ground  height  (height  of  the  ground  and 
the  objects  on  it)  can  be  calculated.  The  critical  issue  of  matching  points  on  both  the  EO 
and  SAR  images  (which  are  required  to  calculate  Ar)  is  discussed  in  Section  5.2. 


4.  Remapping  Using  DTMs 

If  a  DTM  is  available  then  it  is  possible  to  remove  the  smearing  in  the  range  direction 
due  to  elevation.  Once  the  scene  has  been  corrected  for  elevation,  the  information  from 
multiple  sensors,  which  are  also  registered  to  the  same  nadir  view,  can  be  combined 
without  the  need  for  complicated  high  level  matching  procedures  since  structures 
which  are  at  the  same  location  on  the  ground  will  appear  at  the  same  coordinate  in 
each  sensor.  It  is  unlikely  that  the  resolution  of  the  DTM  will  be  such  that  the 
information  about  structures  in  the  image  will  be  included.  This  means  that  objects  in 
the  different  images  will  be  located  at  the  same  coordinate  only  at  their  bases,  vertical 
structures  will  still  be  smeared  in  the  range  direction  of  the  image,  but  this  is  not 
necessarily  a  problem  since  much  of  the  information  which  can  be  obtained  from  an 
oblique  EO  image  is  this  vertical  structure. 

It  should  be  noted  that  DTMs  are  not  available,  or  do  not  exist  at  an  adequate 
resolution,  in  many  areas  of  interest  at  the  moment,  and  indeed  may  not  in  the  future. 
A  common  accuracy  currently  for  a  DTM  in  Australia  is  10m,  yet  to  be  of  use  the  error 
in  the  DTM  should  be  less  than  the  elevation  of  a  pixel  in  the  oblique  image.  This  is  less 
than  lm  in  most  imaging  systems  that  would  be  used  to  extract  the  type  of  information 
being  sought.  In  situations  where  the  local  relief  was  significantly  greater  than  the 
error  in  the  DTM  then  there  is  still  some  value  in  correcting  using  the  DTM,  since  this 
provides  a  coarse  registration  of  the  images;  however  in  this  situation  a  dense  grid  of 
digital  elevations  is  required. 

The  use  of  a  DTM  to  correct  registration  to  allow  for  perspective  variations  in  the 
imagery  would  allow  the  association  procedure  between  sensors  to  be  automated, 
since  objects  would  appear  at  the  same  location;  however,  the  existence  of  DTMs  at  the 


3  It  should  be  noted  that  the  elevation  of  either  the  top  or  bottom  of  the  building  can  be 
determined  to  an  accuracy  controlled  by  the  least  accurate  of  the  SAR  and  EO  sensors,  the 
differential  accuracy  is  controlled  by  the  EO  sensor  alone  since  the  top  and  bottom  of  the 
building  will  appear  at  the  same  location  in  the  SAR  image  so  that  when  the  elevation  of  the 
bottom  of  the  building  is  removed  from  the  top,  the  error  caused  by  the  SAR  cancels,  leaving 
only  the  error  caused  by  the  EO  sensor,  which  in  general  will  be  more  accurate  than  the  SAR. 
Appendix  A2  discusses  the  resultant  errors  that  will  occur  in  this  procedure. 
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required  accuracy  is  unlikely  and  means  that  correction  with  a  DTM  can  provide  little 
more  than  a  coarse  registration. 


5.  SAR  And  Oblique  EO  Image  Fusion 

Long  range  EO  imagery  provides  a  great  deal  of  information  to  an  analyst,  yet  the 
capture  of  this  information  is  subject  to  the  weather,  in  particular  clouds.  SAR  on  the 
other  hand  provides  less  detail  but  will  work  in  all  weather  conditions.  SAR  is  also 
very  good  at  picking  out  man  made  features.  A  dual  sensor  system  consisting  of  SAR 
and  EO  sensors  will  therefore  provide  information  when  for  one  reason  or  another  the 
other  sensor  does  not  work;  however,  more  information  is  available  when  the 
information  from  the  sensors  is  combined.  Three  types  of  information  which  may  be 
deduced  from  fusion  of  SAR  and  oblique  EO  imagery  are  discussed  in  this  section  - 
extraction  of  a  DTM,  structure  extraction  after  elevation  correction,  and  improved 
classification. 

5.1.  Extraction  of  a  DTM 

If  there  is  no  DTM  available  then  the  mis-registration  of  the  SAR  and  the  oblique  EO 
image  gives  the  elevation  of  features  which  can  be  uniquely  identified  in  the  two 
images.  In  all  but  the  simplest  cases  an  automatic  procedure  for  uniquely  matching 
features  in  the  two  images  is  difficult  (see  Section  5.2);  however  it  can  generally  be 
performed  manually  without  difficulty.  An  exception,  where  automated  procedures 
will  work,  is  in  a  scene  with  only  a  few  easily  identifiable  objects,  such  as  trees  or  road 
intersections,  which  can  be  automatically  detected  in  both  sensors.  If  the  objects  are  too 
sparse,  then  the  elevation  of  only  a  few  points  can  be  determined,  if  the  objects  are  too 
close,  then  it  will  become  difficult  to  automatically  associate  a  point  from  one  image 
uniquely  with  a  point  from  the  other  image. 

Elevations  can  thus  be  determined  at  a  number  of  points  through  out  the  image  and 
other  points  can  be  interpolated  between  these  points,  allowing  most  of  the  smearing, 
due  to  elevation,  to  be  removed.  Some  features  with  elevation,  such  as  buildings,  will 
remain  smeared  since  it  is  not  possible  to  find  a  one  to  one  association  between  points 
in  the  images.  The  vertical  facets  of  buildings  and  such  structures  are  smeared  in  the 
EO  image  producing  multiple  features,  while  in  the  SAR  image,  because  the  facet  is 
vertical,  all  those  features  occur  at  approximately  the  same  range  and  consequently 
overlap.  The  result  is  a  many  to  one  association.  A  method  for  determining  the 
elevation  of  these  types  of  structures  is  presented  in  the  next  section.  It  is  assumed  that 
the  correction  determined  in  this  section  has  been  applied  so  that  the  bases  of  these 
vertical  structures  are  positioned  at  the  correct  range.  The  determination  of  DTMs  is 
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probably  more  easily  determined  using  an  interferometric  SAR  so  more  work  in  this 
area  has  not  occurred. 

5.2.  Structure  Extraction  after  Elevation  Correction  using  SAR  and 
EO 

It  is  possible,  by  analysing  an  oblique  EO  image,  to  extract  information  about  the  size 
and  shape  of  objects  in  the  scene.  This  can  be  carried  out  by  recognising  the  type  of 
object  and  the  way  in  which  the  geometry  of  the  imaging  will  depict  that  object,  i.e. 
recognising  an  object  as  a  building,  as  opposed  to  a  road,  changes  the  meaning 
attributed  to  distance  in  the  picture.  Both  distances  are  measured  along  the  same  axis; 
however,  since  we  know  which  edges  are  vertical  and  which  are  horizontal,  the  scale 
applied  to  different  edges  will  vary.  Automation  of  this  process  to  extract  structure  is 
difficult  since  objects  within  the  scene  need  to  be  classified  first  and  then  fitted  to  a 
model  of  that  structure  to  determine  whether  facets  of  the  object  are  vertical,  or 
horizontal,  so  that  the  dimensions  can  be  determined.  In  a  situation  where  large  simple 
buildings  are  of  interest,  fitting  models  to  objects  within  the  scene  can  be  performed 
relatively  simply,  but  the  existence  of  small  vertical  objects  such  as  posts,  towers,  small 
buildings,  and  complex  objects  such  as  buildings  on  wharfs,  would  make  this  approach 
difficult,  so  an  alternative  has  been  sought. 

An  alternative  to  fitting  models  to  the  scene,  is  possible  when  the  additional 
information  from  a  SAR  image  is  available.  The  oblique  EO  sensor  determines  the 
existence  of  facets  by  detecting  the  boundaries  of  uniform  regions.  The  SAR  image 
enables  us  to  determine  the  slope  of  some  of  the  facets,  since  horizontal  to  rising 
vertical  facet  boundaries  produce  a  strong  return,  while  horizontal  facets  produce  a 
much  lower  response  (this  can  be  seen  in  Figure  11  with  the  buildings  around  the 
wharf).  The  SAR  information  needs  to  be  matched  to  the  information  in  the  EO  image  if 
it  is  to  be  used.  In  the  following  example  the  oblique  EO  image  has  already  been 
distorted  so  that  the  ground  is  registered  to  the  SAR  image,  this  means  that  matching  is 
simple  because  the  base  of  objects  will  occur  at  the  same  range  and  crossrange  in  the 
SAR  and  registered  EO  image.  Structures  with  elevation  within  the  EO  image  are  still 
smeared  in  range;  the  determination  of  their  structure  will  enable  this  smearing  to  be 
removed. 

It  is  assumed  that  different  facets  of  a  building  would  be  separated  by  a  discontinuity 
in  the  intensity  map  in  the  EO  image.  The  transition  from  a  horizontal  surface  to  a 
vertically  rising  facet  of  a  building  produces  a  line  in  the  SAR  image,  while  other 
transitions  will  probably  produce  little  effect.  Structures  with  vertical  faces  such  as 
buildings  were  also  assumed  to  have  the  same  shaped  edge  at  the  top  of  the  vertical 
face  and  at  the  bottom,  (ie.  vertical  faces  are  rectangles)  Figure  6  shows  an  example  of  a 
building's  facets  as  seen  in  a  registered  oblique  EO  image  (top),  the  same  building  as 
seen  by  a  SAR  sensor  (shown  beneath  the  EO  image  -  note  that  only  the  leading  bottom 
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edges  are  visible),  and  the  cross  section  of  the  building  after  reconstruction.  The  images 
in  Figure  6  were  acquired  from  the  left  side  of  the  page,  where  the  aircraft  is  shown,  so 
range  is  measured  from  left  to  right,  and  cross  range  from  the  top  to  the  bottom  of  the 
page.  The  different  facets  of  the  buildings  are  identified  from  the  EO  scene  by  their 
different  colours.  In  the  SAR  image  of  the  equivalent  scene,  only  the  bottom  left  facing 
edges  can  be  seen. 

Discontinuities  in  the  EO  image  gradient  reflect  a  change  in  the  angle  of  the  reflecting 
surface.  Facet  edges,  within  the  oblique  optical  scene,  which  have  a  component 
perpendicular  to  the  line  of  sight  are  determined  by  differentiating  the  intensity  along 
each  line  of  pixels  in  the  range  direction  (from  left  to  right  in  Figure  6).  By  scanning  the 
EO  and  SAR  images  along  a  line  of  pixels  which  have  the  same  cross  range,  ie.  from  left 
to  right  in  Figure  6,  the  bottom  edge  of  vertical  facets  of  buildings  will  be  revealed 
when  the  line  of  pixels  being  scanned  cross  a  line  in  the  SAR  image  and  a  facet  edge 
indicated  by  a  discontinuity  of  the  intensity  in  the  EO  image  ('First  Edge'  in  Figure  6). 
The  end  of  the  vertical  facet  will  be  the  next  discontinuity  encountered  in  the  EO  image 
('Second  Edge'  in  Figure  6)  and  the  next  facet  will  be  horizontal.  This  horizontal  facet 
continues  until  another  discontinuity  is  detected  ('Third  Edge'  in  Figure  6)  which 
represents  the  opposite  edge  of  the  building. 

To  improve  the  performance  of  the  above  approach  and  to  prevent  spurious  edges 
from  being  used  for  the  second  and  third  edges  of  a  building,  which  would  generate 
false  structure  information,  the  selection  of  the  second  and  third  edges  is  constrained  to 
discontinuities  in  the  EO  image  which  are  parallel.  This  is  true  for  rectangular 
buildings  (refer  to  the  cross  section  line  C-D  in  Figure  6)  except  near  the  edges  (refer  to 
the  cross  section  line  A-B  in  Figure  6)  where  it  is  still  true  for  the  second  edge;  however 
the  horizontal  top  facet  of  a  building  may  have  front  (second  edge)  and  back  edges 
(third  edge)  on  the  same  cross  range  line  which  are  perpendicular. 
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Figure  6:  Matching  of  edges  in  the  SAR,  EO  and  Reconstructed  object. 


This  simple  procedure  is  capable  of  determining  the  size  and  position  of  simple 
buildings  in  a  scene.  By  adding  thresholds,  and  matching  edges  which  are  parallel,  so 
that  only  significant  edges  are  detected  the  process  will  also  work  with  small  amounts 
of  additive  noise.  With  increased  amounts  of  noise  the  procedure  will  periodically  miss 
an  edge  or  generate  another  edge  which  makes  all  successive  facets  incorrect.  By 
limiting  the  potential  size  of  buildings  to  a  reasonable  size  this  kind  of  error  can  be 
detected  and  the  process  can  be  placed  back  on  track. 

This  procedure  could  be  improved  markedly  by  incorporating  additional  knowledge 
about  the  expected  shape  of  objects;  however,  this  would  mean  that  the  analysis  would 
become  more  and  more  specialised  as  its  performance  improved. 

To  illustrate  the  operation  of  the  structure  extraction  procedure  a  simulation  was 
performed  with  a  simple  scene  consisting  of  a  number  of  buildings.  The  registered 
oblique  EO  image  and  the  SAR  image  are  shown  in  Figure  7.  Illumination  of  the  EO 
image  is  coming  from  behind  the  aircraft,  and  a  simple  reflectance  model  has  been 
utilised  to  determine  the  color  of  the  facets.  Because  the  illumination  source  is  coming 
from  behind  the  aircraft,  shadows  from  the  buildings  are  obscured.  The  first  subfigure 
in  Figure  7  represents  the  simulated  registered  oblique  EO  scene  with  additive  white 
gaussian  noise,  the  second  is  the  SAR  image  with  noise  (the  noise  has  a  Weibull 
distribution  [2]). 
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Figure  7:  Simulation  of  structure  extraction,  registered  oblique  EO  image,  and  SAR  image. 


Figure  8  contains  two  images,  the  first  depicts  the  elevation  which  was  derived  using 
the  above  mentioned  procedure,  with  an  intensity  corresponding  to  the  elevation,  at 
each  point  in  the  EO  image.  The  height  of  an  object  in  the  EO  image  can  be  determined 
by  examining  the  intensity  of  the  corresponding  pixel  in  the  elevation  image.  The 
second  subfigure  in  Figure  8  represents  the  unsmeared  EO  image  determined  using  the 
calculated  elevation  information.  This  final  image  shows  where  the  mapping  has  and 
has  not  been  successful.  In  particular  the  procedure  has  failed  at  the  corners  of 
buildings  where  the  edges  of  the  top  facet  on  the  same  horizontal  line  (line  of  constant 
cross  range)  are  not  parallel  but  perpendicular,  since  they  correspond  to  adjacent 
edges,  not  opposite  edges  as  assumed.  Where  the  mapping  has  been  performed 
successfully  only  the  tops  of  buildings  should  appear  since  their  vertical  faces  are  not 
seen  from  above.  The  approach  has  generated  the  correct  height  of  all  the  buildings,  in 
particular  the  top  three  buildings  have  had  almost  all  the  smearing  in  range  removed. 
Because  of  the  orientation  of  the  lowest  building,  the  third  edge  is  almost  always 
perpendicular  to  the  first  two  edges,  so  the  elevation  of  the  vertical  sides  has  been 
determined  accurately  (and  mapped  to  a  line  in  the  second  subfigure  of  Figure  8  as 
required)  but  the  horizontal  roof  of  the  building  has  not  been  determined. 


Figure  8:  Results  of  structure  determination  using  information  from  Figure  7.  Elevation  of 
points  in  the  oblique  EO  image,  and  the  resultant  image  after  the  smearing  in 
range,  caused  by  the  object's  structure,  has  been  removed. 


14 


UNCLASSIFIED 


UNCLASSIFIED 

DSTO-TR-0117 

5.3.  Classification  after  Elevation  Correction  using  SAR  and  EO 

Whether  the  oblique  EO  image  is  elevation  corrected  using  a  DTM,  or  whether  features 
have  been  matched  to  form  a  DTM  which  has  then  been  used  to  correct  the  image  is 
unimportant.  After  elevation  correction  the  SAR  and  EO  images  should  be  registered. 

The  smearing  in  range  of  vertical  structures  need  not  be  corrected  since  the  region 
concealed  by  the  smearing  corresponds  to  a  region  of  shadow  in  the  SAR  image.  The 
two  registered  images  can  be  used  as  inputs  to  a  classifier  to  increase  the  information 
available  for  decision  making  processes,  thus  improving  the  classification  performance 
of  a  single  sensor  alone. 

The  two  sensor  systems  once  registered  provide  different  information  which  would 
make  classification  of  objects  in  the  scene  easier.  SAR  is  good  at  picking  man  made 
objects  from  natural  objects  and  could  at  the  very  worst  draw  attention  to  man-made 
objects  within  an  optical  scene. 

Clement  et.  al.  [3],  Wong  and  Orth  [4],  Liu  et.  al.  [5]  and  JPL  Pasadena  [6],  have 
demonstrated  that  it  is  possible  to  successfully  register  SAR  images  with  multispectral 
data,  and  that  the  results  obtained  are  beneficial  for  the  purpose  of  classification  and 
analysis;  however  in  the  above  examples  registration  required  the  manual  extraction 
and  matching  of  control  points  from  the  two  images. 


6.  Difference  Detection  After  Elevation  Correction 

Has  Occurred 

In  the  normal  mode  of  operation  the  effective  azimuth-elevation  projection  of  the  SAR 
is  similar  to  a  nadir  view,  particularly  for  relatively  flat  scenes  as  is  common  in  a  large 
portion  of  remote  Australia.  Because  the  view  is  almost  a  nadir  view,  images  taken  at 
different  times  from  slightly  different  locations  will  appear  to  have  the  same  geometry 
(ie.  objects  will  not  appear  to  be  distorted  between  the  scenes).  Objects  in  EO  images 
are  far  more  sensitive  to  elevation  than  objects  in  a  SAR  image,  and  so  images  of  objects 
taken  from  slightly  different  locations,  will  be  distorted  and  located  in  different 
positions  within  the  scene,  in  all  but  perfectly  flat  terrain,  making  the  detection  of 
change  between  two  images  almost  impossible.  If  a  SAR  and  an  oblique  EO  image  are 
captured  simultaneously  and  it  is  possible  to  associate  objects  in  the  SAR  with  objects 
in  the  EO  imagery,  then  the  distortion  in  the  oblique  EO  image  caused  by  the  terrain 
can  be  removed  and  the  EO  image  can  be  aligned  with  other  EO  images  captured  at 
different  times  so  that  difference  detection  can  occur.  This  procedure  is  superior  to 
undistorting  images  using  a  DTM,  because  the  resolution  of  DTMs  is  low,  and  the 
elevation  information,  at  least  at  the  moment,  is  only  known  on  a  sparse  grid.  Features 
in  the  EO  image  can  be  corrected  using  the  DTM  but  the  correction  suffers  because  of 
low  accuracy  in  the  DTM  values  and  from  interpolation  errors  between  the  DTM  grid 
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points,  whereas  if  the  elevation  is  corrected  using  a  simultaneously  acquired  SAR 
image  then  the  objects  of  interest,  which  are  seen  by  each  sensor,  can  be  located  at  the 
precision  of  the  sensors,  allowing  the  object  and  its  location  to  be  compared  between 
images  at  different  times. 


7.  Resolution  Issues 

Until  this  point  a  discussion  of  resolution  has  been  avoided,  so  as  not  to  further 
complicate  the  previous  discussions.  In  general  the  EO  sensor  will  have  a  higher 
resolution  than  the  SAR  sensor,  but  the  issue  is  not  that  simple  either.  If  the  EO  sensor 
has  square  pixels  then  the  projection  on  the  ground  of  each  pixel  will  be  far  from 
square  so  that  the  resolution  in  the  range  and  cross  range  will  be  different.  In  the 
example  images  presented  the  range  resolution  of  the  EO  projection  was  the  same  as 
the  range  resolution  of  the  SAR,  while  the  cross  range  resolution  was  much  higher  than 
in  the  SAR.  Different  SAR  products  (such  as  a  single  look  image)  would,  however,  have 
a  different  cross  range  resolution. 

In  registering  and  displaying  the  images  it  is  necessary  to  either  reduce  the  resolution 
of  some  of  the  image  components,  or  to  increase  the  resolution  of  other  image 
components  by  interpolating  so  that  the  range  and  cross  range  resolution  of  all  the 
images  are  identical.  The  result  of  course  is  to  either  destroy  information  or  to  generate 
much  larger  images  which  may  then  result  in  storage  difficulties. 

The  consequence  of  the  various  resolutions  is  that  elevation  information  obtained 
from  the  fusion  of  the  images  will  have  a  different  resolution  depending  upon  the 
mechanism  by  which  the  information  was  extracted.  By  matching  a  feature  in  the  SAR 
and  the  EO  image  height  is  determined  which  has  a  resolution  which  is  a  function  of 
the  resolution  of  the  SAR  range  resolution  and  the  projected  EO  range  resolution,  while 
if  the  height  is  determined  from  range  smearing  in  the  EO  sensor  then  the  height 
resolution  is  close  to  the  elevation  resolution  of  the  EO  sensor  rather  than  the  projected 
range  resolution. 


8.  Conclusion 

Elevation  information  can  be  obtained  by  fusing  a  SAR  and  an  oblique  EO  image  -  this 
information  cannot  be  obtained  from  either  of  the  sensors  individually.  Unfortunately 
this  information  requires  objects  within  the  images  to  be  identified  uniquely,  and  with 
the  exception  of  some  simple  scenes  it  is  unlikely  that  this  identification  problem  can 
be  solved  automatically.  In  the  situations  where  an  automatic  solution  is  possible  it 
may  still  be  more  efficient  to  perform  the  operation  manually  given  the  effort  required 
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to  set  up  the  automation  procedure  and  the  limited  versatility  and  area  of  applicability 
of  the  automation. 

The  registration  of  the  oblique  EO  image  to  a  plan  map  or  other  sensor  is  certainly  of 
significance  to  aid  in  the  use  of  several  sources  of  information,  since  it  means  that  the 
information  appears  the  same,  and  less  time  need  be  spent  looking  for  the 
corresponding  information  from  the  multiple  sources.  Unfortunately,  measuring  the 
camera  and  line  of  sight  parameters  to  perform  this  operation  is  not  a  simple  task  and 
registration  may  need  to  be  done  by  the  detection  of  control  points  with  a  known 
elevation. 

For  oblique  EO  and  SAR  information  to  be  fused  to  improve  classification,  accurate 
registration  which  takes  into  account  the  height  of  features  in  the  scene  must  occur. 

Since  DTM  are  not  available  at  the  appropriate  resolutions  the  elevation  of  features 
must  be  determined  manually.  The  question  needs  to  be  raised,  whether  the  effort  in 
registering  the  images  is  greater  than  that  required  for  manual  analysis  and 
classification. 

The  fusion  of  SAR  and  oblique  EO  images  is  useful  as  an  intelligence  gathering  tool 
which  will  enable  users  to  remove  distortions  from  oblique  images  and  determine  any 
structural  changes  in  the  scene  which  may  have  occurred  between  the  acquiring  of  the 
EO  images,  such  as  the  movement  or  change  in  size  of  objects. 

The  extraction  of  elevation  information  is  also  possible;  however  this  process  will  be 
difficult  to  automate  and  is  consequently  not  a  viable  method  of  determining 
elevations  over  a  large  area. 
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Appendix  A 


Calculations. 

A1  Derivation  of  Registration  Errors 

The  errors  in  range  and  cross  range  caused  by  small  errors  in  determining  the 
position  and  orientation  of  the  camera  are  approximated  by  the  following  expressions, 

a  a  Jra  .  ,  dra  ..  dra 

Ara  ~  A az..  - +  A0  — —  +  A dec - +  Ah - 

dazc  90  ddec  dh 

a  a  dcra  dcra  .  ,  dcra  . ,  9  era 

A  era  ~  A  az.c  - —  +  A0  — —  +  A dec - +  Ah - 

dazr  99  ddec  dh 
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-  =  rcos(azc  +  az) 
=  dist  -  era 
■  =  rsin(azc  +  az) 


dra  dr  daz 

"aeT  =  90  Sin  +  ~  r— cos(azr  +  az) 

dr  del  dy  .  daz  dx  ,  . 

=  M + a7)  +  aTae (c™  “ d,sl) 


-rfxs\n(azc  +az ) 


fy(cra  -  dist ) 


if2  +  y2  )sin(dec  -  el)cos(dec  -  el)  f2  +x 2 

dcra  dr  ,  daz  .  , 

—  =  --CO  S(azr  -azy-r—s^az,  +  az) 

dr  del  dy  ,  x  daz.  dx 

=  'cos (azc  +az. ) — — — ra 

del  dy  dQ  dx  90 

_ _ rfx  cos(azc  +az) _  fyra 

(f2  +  y2  )sin  (dec  -  el)  cos  (dec  -  el)  f2  +  x2 
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dra  dr 


sin(«zr  +  az.) 


ddec  ddec 

-rs\n(az.c  +  az.) 
s\n(dec  -  el)cos(dec -  el) 
dcra  dr 

— —  =  -——cos  (az,  +  az) 
ddec  ddec 

_  rcos(azr  +  az.) 

sin(dec  —  el)cos(dec  —  el) 


dra  dr  .  . 

—  =  7-sm  (az,.  +  az.) 
dh  dh 

sin(«zf  +  az.) 

tan  (dec  -  el) 

dcra  dr 

=  -—-cos(az.r  +  az) 
dh  dh 

cos  {az.,.  +  az.) 

tan  (dec  -  el) 


Ara  ~  Aaz.c{dist  -  era)  +  A0 


-rfxsm(azc  +az) 


fy(cra  -  dist) 


(/2  +  y2)sin(dec  -  el)cos(dec  -  el)  f 2  +  x~ 


,  ,  [  -rsin(az.r  +  az)  )  ..[sin (az.c+az) 

+Adec  - - .  +  An  - - — - 

sin  (dec  -  el)  cos  {dec  -el) )  ^  tan  {dec  -  el) 


Acra  ~  Aaz.Ara)  +  A0 


rfxcos(az.c  +  az.)  fyra 

(  f2  +  y1  )sin {dec  -  el)cos(dec  -  el)  f2  +  x1 


AI  [  rcos.(azr+az)  )  . ,  f  cos(az.r  +  az.) 

+Adec  - ^ - - -  +  Ah  - - — ^ - - 

.sin(dec  -  el)cos(dec  —  el)J  ^  tan  (dec  -  el) 


Since  era  and  ra  are  large  compared  to  the  other  parameters  in  the  above  expression 
the  error  in  ra  and  era  {Ara  and  Acra)  will  be  particularly  sensitive  to  errors  which  are 
multiplied  by  ra  or  era.  In  particular  Ara  and  Acra  are  extremely  sensitive  to  the  angular 
measurement  errors  Aazc,  A0  and  Adec  in  the  long  range  oblique  system  under 
investigation.  For  example,  if  the  resolution  of  the  imaging  sensor  is  lm  and  the  range 
is  approximately  30km  then  it  is  required  that  the  measurement  error,  A azc ,  be  less 

than  1/30000  radians  (~33prad),  if  the  cross  range  estimate  is  to  be  comparable  to  the 
sensor  resolution.  The  accuracy  of  this  measurement  would  be  difficult  and  unrealistic 


UNCLASSIFIED 


UNCLASSIFIED 

DSTO-TR-0117 

to  obtain;  however,  there  are  some  potential  solutions  to  the  problem  which  are 
outlined  in  Section  2.1. 

Although  it  is  possible  to  determine  the  relative  position  of  the  EO  sensor  to  the  SAR, 
particularly  if  they  are  mounted  on  the  same  stabilising  platform  (stabilisation  of  EO 
platforms  to  1 7urad  is  possible),  the  errors  will  effect  the  two  sensors  in  different  ways. 

For  example  since  the  SAR  measures  range  and  azimuth,  the  errors  A az.c ,  A0  and 

A  dec  will  not  affect  the  range  or  cross  range  positions  of  objects  with  in  the  scene.  The 
appearance  and  resolution  of  the  SAR  image  may  be  altered  but  the  location  of  objects 
in  the  scene  will  not  be  altered. 


A2  Derivation  of  Height  Estimation  Errors  Although  the  absolute  value 
of  the  camera  parameters  cannot  be  determined  to  a  sufficient  accuracy  to  allow  the 
determination  of  the  correct  range  and  cross  range  positions  in  the  image,  the  question 
remains  over  whether  the  parameters  are  well  enough  known  that  the  relative 
positions  of  objects  in  the  scene  can  be  determined.  To  determine  the  height  of  an 
object,  such  as  a  building,  it  is  necessary  to  know  the  different  ranges  at  which  the  top 
and  bottom  of  the  object  appear. 


The  errors  involved  are  analysed  below  and  it  is  found  that  the  error  is  approximately 
proportional  to  the  height  of  the  object.  These  errors  are  lower  than  the  errors  in  the 
absolute  values  of  range  and  cross  range;  however  for  large  objects  the  error  may  still 
be  significant. 
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f  dra, 
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f 
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/  _  1 


..0  0  y.O  O 

,/-+x-  /-+r  ./ 

for  small  structures 


:  and  sin  {dec  -  el2 )  cos  (dec  -el2)~  sin  (dec  -  el{ )  cos  (dec  -  e/, ) 


dra ,  dra ,  .  ,  (  r,  r. 

— L  _  — -  =  sin(«zr  +  az.)  - = - 

ddec  ddec  \sin(dec-el)cos(dec-el)  sm(dec  -  el)cos(dec  -  el) 


sin(az,.  +  az) 

sin  {dec  -  el)cos(dec  -  el) 


(r2 


dra,  dra0  .  ,  J  1  1 

— — - — —  =  sm(azr  +  az) - 

dh  dli  i^tan  (dec-el^)  tan  (dec-el2) 


sin(azt.  +  az) 


('i-'O 


tan  {dec-el^)  h 


A height  °c  A azJU  -  /;  )cos(az  +  az))  +  A0  - ■*sin(flz^  +  az) - /  _  \ 


v  7  ^/sin(<iec-«/)cos(Jec-e/) v ' 

(era  -  dist)  /  0  .  ,  (  sin(azr  +  az)  /  \ 

+  AA(Sin(^+aZ)(^-ra)) 

Ac/z,  £a, (/',  -  r2)+ A9  /c9^  -r2)+  Adec ^ (r,  -  r2^+ Ah kh (r,  -  r2j 


assuming  that  (y,  -  y2 )  °c  (r,  -  r2 ) . 


The  above  expression  shows  that  the  error  in  the  estimation  of  the  height  of  an  object  is 
approximately  proportional  to  the  height  of  the  object,  since  the  height  of  an  object 
results  in  a  different  range  between  the  base  and  the  top.  The  proportionality  constant 
is  difficult  to  evaluate,  and  varies  over  a  scene,  but  typical  values  for  k  are  less  than  1. 
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Appendix  B 
Images. 


Figure  9:  Oblique  image  of  a  region  near  Port  Adelaide. 
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Figure  10:  Transformed  oblique  image  from  Figure  9.  (registered  to  Figure  11) 
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Figure  11:  SAR  image  of  the  same  region  as  Figure  9  and  10. 
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Figure  12:  Oblique  image  of  a  region  near  Edinburgh  airbase. 


Figure  13:  Oblique  1R  image  of  a  region  near  Edinburgh  airbase 
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Figure  14:  Transformed  oblique  image  from  Figure  12.  (registered  to  Figure  16) 


Figure  15:  Transformed  oblique  IR  image  from  Figure  13.  (registered  to  Figure  16) 


Figure  16:  SAR  image  of  the  same  region  as  Figure  14  and  15. 
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Appendix  C 

Proposed  Future  Work. 


It  is  possible  to  determine  the  size  of  objects  in  an  oblique  EO  image  if  we  already 
know  what  the  structure  of  the  object  is  and  the  way  in  which  different  parts  of  the 
object  will  appear.  The  human  brain  is  able  to  examine  these  oblique  images  and 
determine  the  size  and  shape  of  objects  since  we  can  also  usually  determine  what  the 
objects  are.  For  this  process  to  be  automated  the  task  is  somewhat  harder;  however,  if 
the  range  of  objects  of  interest  is  reduced,  and  additional  clues  are  provided,  then  the 
task  becomes  more  manageable. 

It  is  proposed  to  use  images  obtained  from  a  SAR  and  an  oblique  EO  sensor  to 
determine  the  size,  structure  and  position  of  objects  on  the  ground.  This  procedure  will 
involve  several  well  defined  steps: 

1.  Approximate  flat  earth  registration.  Using  an  assumption  that  the  earth  is  flat  and 
knowing  the  camera  parameters,  distort  an  oblique  EO  image  to  align  with  a  SAR 
image. 

2.  Process  images  to  extract  features. 

3.  Identify  objects,  i.e.  man  made  objects  should  be  clearly  visible  in  the  SAR  image. 

4.  Match  features  in  the  two  images,  i.e.  lines  in  the  same  location  of  the  same  length 
and  orientation  are  probably  caused  by  the  same  feature. 

5.  Infer  structure  from  matched  features  and  probable  hypothesis  about  object  type, 
i.e.  A  building  has  only  a  limited  number  of  possible  structures,  the  matching  of 
features  should  allow  the  other  features  to  be  easily  determined. 
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